Images obtained by image pickup apparatuses such as digital cameras are degraded in quality by blurring. Blurred images are caused by spherical aberration, coma aberration, curvature of field, astigmatism, and the like of image pickup optical systems. The above aberrations can be represented by the point spread function (PSF, Point Spread Function). An optical transfer function (OTF, Optical Transfer Function) that can be obtained by the Fourier transform of the point spread function (hereinafter, PSF) represents information about the aberrations in a frequency space. The optical transfer function (hereinafter, OTF) can be represented by the complex number. The absolute value of the OTF, that is, the amplitude component, is called the MTF (Modulation Transfer Function), and the phase component is called the PTF (Phase Transfer Function). Here, the phase component is represented as the phase angle using Equation 1 below. Re(OTF) and Im(OTF) are the real part and the imaginary part of the OTF, respectively.PTF=tan−1(Im(OTF)/Re(OTF))
In this manner, the optical transfer function of the image pickup optical system causes degradation in the amplitude component and phase component of an image, and in a degraded image, therefore, each point of an object is asymmetrically blurred as with coma aberration. Furthermore, chromatic aberration of magnification is caused by acquiring, as, for example, RGB color components, shifted image forming positions caused by the difference in imaging magnification from one wavelength of light to another, in accordance with the spectral characteristics of the image pickup apparatus. Accordingly, image spread is caused not only by a shift in image forming position between R, G, and B but also by a shift of an image forming position from one wavelength to another within each color component, that is, a phase shift. Thus, when the point spread function is viewed in a one-dimensional cross section in each direction (azimuth direction) perpendicular to the principal ray (light ray propagating through the center of the pupil of the image pickup optical system), aberration-induced degradation of the phase component (phase degradation component) causes asymmetry in the point spread function. Further, degradation of the amplitude component (amplitude degradation component) has an influence on the spread size of the PSF in each azimuth direction.
Therefore, in order to accurately correct for image degradation caused by the image pickup optical system using image processing, it is necessary to correct the aberration-induced phase degradation component and the amplitude degradation component.
Further, a known method for correcting for the amplitude degradation component and the phase degradation component is to perform correction using information about the optical transfer function (OTF) of the image pickup optical system. This method is called image restoration or image recovery, and a process for correcting (reducing) the degradation component of an image using the information about the optical transfer function is hereinafter referred to as an image restoration process.
An overview of the image restoration process will be described hereinafter.
When a degraded image is represented by g(x, y), the original image is represented by f(x, y), and the point spread function (PSF) of an image pickup optical system that is used to acquire g(x, y) is represented by h(x, y), the equation below holds true, where * denotes convolution (convolution integral, sum of products) and (x, y) denotes the coordinates of an image in the real space.g(x, y)=h(x, y)*f(x, y)   (Equation 1)Converting the Fourier transform of Equation 1 into a display format in the frequency space yields the expression of Equation 2.G(u, v)=H(u, v)·F(u, v)   (Equation 2)Here, H(u, v) is the optical transfer function (OTF) that is the Fourier transform of the point spread function (PSF) h(x, y). G(u, v) and F(u, v) are the Fourier transforms of g(x, y) and f(x, y), respectively. (u, v) denotes the frequency (coordinates) in a two-dimensional frequency space. The initial image (original image) may be obtained from the degraded image by dividing both sides of Equation 2 by H(u, v).G(u, v)/H(u, v)=F(u, v)   (Equation 3)By returning the inverse Fourier transform of F(u, v), that is, G(u, v)/H(u, v), to the real space, the original image f(x, y) can be obtained as a restored image.
By taking the inverse Fourier transform of both sides of Equation 3, Equation 3 is expressed as Equation 4.g(x, y)*R(x, y)=f(x, y)   (Equation 4)Here, the inverse Fourier transform of 1/H(u, v) is represented by R(x, y). R(x, y) is an image restoration filter.
Since the image restoration filter is based on the optical transfer function (OTF), degradation of the amplitude component and the phase component can be corrected for.
A Wiener filter capable of controlling amplification of noise is known as the image restoration filter. The Wiener filter is an image restoration filter that changes the degree of restoration in accordance with the intensity ratio (SNR, Signal to Noise Ratio) of an input signal (image signal) to a noise signal in order to reduce noise in the input image.
Further, PTL 1 discloses an image restoration filter that is an applied Wiener filter in which an image restoration filter has an adjustment coefficient α. By adjusting the parameter α, the image restoration filter is capable of changing the degree of restoration of an image in a range from a filter that outputs an input image as it is (filter that is not applied to an input image) to a filter that maximally performs image restoration (inverse filter).
Further, PTL 2 discloses an edge enhancement process as a method for correcting for phase degradation, in which an edge portion of an image is detected and the edge is enhanced.